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ABSTRACT 



A science classroom was used to study the effects of a 
shared intranet environment on students* problem-solving ability and 
metacognitive reflection skills through shared contextualization. Subjects 
were first-time 9th- and lOth-grade biology students from three public high 
schools. An evaluative analysis of the Courseinfo software (Blackboard, Inc., 
1998) was undertaken, and the tracking capabilities of the Courseinfo 
software were evaluated through measurement of: number of log-ons to the 
threaded discussion; number of threaded statements; number of threaded 
dialogue statements of response to other student statements; and number of 
threaded dialogue statements of response to teacher statements. Analysis of 
tracked user movement within the shared Web -based environment was 
investigated as one possible model of evaluating learner academic behaviors 
within, and as a result of, Web-based environments. Results of the study 
support tracking user movement as an avenue of quantifying the effectiveness 
of Web-based learning initiatives. (AEF) 
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Introduction and Rationale 

As the intranet/Internet is introduced in to the learning environment of the classroom today, the effects of 
this technological environment must begin to be studied. Technology offers an opportunity to affect and 
monitor academic behaviors such as problem-solving ability and metacognitive reflection. However, what 
evidence can be found that speaks to the effectiveness of the use of such learning places? Can a model for 
visualizing any level of effectiveness be generated for such environments? 

As technology creates a virtual classroom environment as it moves to a Web-based space, questions arise 
regarding the effectiveness of such geographically unrestricted, collaborative problem-solving places 
Qacobsen & Levin, 1993) . Collaborative designs such as these become possible when the learning 
environment is placed within a Web of computers, thereby facilitating and encouraging access by many to a 
shared intranet/Internet Web-place. 

As students use the collaborative capabilities of a networked intranet learning environment, thinking about 
their own thinking evolves, thereby increasing the opportunity to clarify misconceptions of knowledge, 
procedural or declarative. The science classroom presents one opportunity to study the effects of a shared 
intranet environment on student problem-solving ability and metacognitive reflection skills through shared 
contextualization. Field study data collection, posting, and discussion create a context, or anchor (Brown et 
al, 1989), for this virtual learning environment based on real, concrete information. 



Questions Researched 

As part of a secondary research question, an evaluative analysis of the Courseinfo software (Blackboard, Inc., 
1998) was undertaken. The tracking capabilities of the Courseinfo software were evaluated through 
measurement of (1) the number of log-ons to the threaded discussion Web page, (2) the number of threaded 
statements, (3) the number of threaded dialogue statements of response to other student statements, and (4) 
the number of threaded dialogue statements of response to teacher statements. 
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Analysis of tracked user movement within the shared Web-based environment was investigated as one 
possible model of evaluating learner academic behaviors within, and as a result of, Web-based environments. 
While further analysis is needed, results of this study support tracking user movement as an avenue of 
quantifying the effectiveness of Web-based learning initiatives. 



Study Background 

Subjects 

Subjects for this study were first-time 9th- and lOth-grade biology students from three public education high 
schools in the Conroe ISD within Conroe, Texas. The sample (n) contained 78 students of the 1,400 students 
enrolled in Biology I courses. Two classes from each school were selected and randomly assigned to a control 
class and a treatment class. Selected campuses operated on an A-B, 90-minute class alternating block 
schedule. 



Technology 

Participants assigned to treatment groups received access to technology. This technology included 
Macintosh® platform computers. Scanners, digital cameras, Internet connections, and laser printers rounded 
out the technology utilized by the treatment groups. Software access included Apple® QuickTake® PhotoNow 
software, HP® scanning software, Microsoft® Office, Netscape Navigator® Gold 3.0, and Inspiration® 4.0. 



Ecology Curriculum 

Teachers at selected campuses received an Adopt-a-Ditch ecology curriculum (Stone & Myers, 1994). This 
modified curriculum provided the context for the Web-based learning environment. Training for all 
curriculum lessons, the LaMotte Freshwater Testing Kits, and the Web-based database was provided. 
Spontaneously generated forum topics were noted as the researcher analyzed collected data within the Web- 
based learning environment. 



Teacher Training 

Technology training consisted of instruction and practice in using the Courseinfo intranet simulation 
software (Blackboard, Inc., 1998), uploading and downloading Inspiration files through the Internet, using a 
digital camera and downloading images, working with Excel spreadsheets and graphics, and using the 
discussion forum environment. 



Software Background 

A secondary intent of this study was to provide evaluative discussion of the software product used to create 
the shared learning environment. As this newly developed distance education software product was in the 
process of being piloted by a variety of institutions and research programs, permission for its use as part of 
this study was obtained (from Blackboard, Inc.). Agreements between the author and software developers 
included evaluative information resulting from use of the product, taking into consideration the constraints 
of its application within the nature of the study. Results addressing the secondary research question focused 
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on an examination of tracked user movement findings, discussion forum findings, and difficulties 
experienced during use of the software product. 

One caveat exists as a part of answering this secondary research question. It is imperative to remember that 
this section’s evaluative basis remains the by-product of this particular study. As many other piloted uses of 
this product have occurred, or are presently ongoing, this author suggests that additional review of other 
evaluative efforts be examined should any f^uture use of the product be initiated. 

Further, it must be noted that use of the product within this study represents a model whereby security access 
was granted to all subjects within the study. This type of widespread access is not the usual mode of access 
suggested by the developers. However, this approach was utilized to create a more constructivist Qonassen, 
1996), shared environment given the nature of the software product. Therefore, review of the secondary 
research question findings should occur with forethought of these caveats. Future users of this software 
product should consider the type of learning environment they envision, as well as the learning philosophy 
underpinning that environment, as this study’s findings of the software are taken into account. 



Software Capabilities within the Model of Study 



Tracked User Movement 

Courseinfo software provides a database of user movement patterns once entry in to the Web site occurs (see 
Figure 1) . Analysis of recorded user movement can be obtained for a variety of statistical views of user 
movement. 
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Figure 1. Announcement page for Web-based learning environment. 



These statistical views were made available through the CourseSite Stats option accessible with a security 
password through the control panel. Views (analyses) available include: (1) traffic hits for the entire course 

time and by links (see Figure 3); (5) traffic hits by countries, visitors, and browsers; and (6) filtering the 
preceding information by server domain. 
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Figure 2. Movement analysis by Web-site page. 
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Figure 3. Analysis by Web-site day and time. 

As a part of the study, tracked user movement generated information related to several aspects of user 
movement. Repeated visits (interpreted as level of traffic) can be connected to user preferences related to 
Web-page function, design, or information. The total number of hits can indicate to some degree user time 
within the shared learning environment in general. Evaluating which pages were accessed during the study 
could indicate depth and type of use of the shared environment. Use of help pages within the shared 
environment could imply use in a self-sufficient way. Finally, tracked use of the discussion board provided 
information as to the development of the entire study group, the collaborative extent of the group, and 
student-initiated discussion. Tracked user movement, identified movement by page title, and number of 
traffic hits, as well as percentages, were summarized (see Figure 4). 

Analysis of user movement by page indicated entrance in to the shared environment through the main page. 
The Course Documents page was used with the most frequency (13.3%) after entering the shared 
environment. This page provided the storage location for uploaded files (graphs, images, and documents) for 
all participants. All files accessed through this page by any participant were viewed or downloaded. Use of the 
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Assignments page incurred 9.8% of the hits. This page held sample files and user instructions for the various 
activity demands of the ecology 
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Figure 4. Sample discussion threads used in tracking learner movements. 



curriculum. The limited use of this page simulated the nature of FAQs pages in other Web sites, that is, it 
remained relatively unused. The third most frequented page within the Web site was the student Tools page, 
at 11.9%, indicating that help pages entitled Tools seem to be more frequently accessed by novice users than 
pages titled in other ways. 



An analysis of user movement by date was undertaken as an indicator of the overall use of the shared 
environment. Table 1 contains a use-by-date display of user movement within the Web site. Examination of 
use-by-date hits, filtered through server domain, provides evidence of the site’s extension of interactivity 
between time and the users. At first glance an appearance of minimal use might be seen in the small number 
of hits and the low percentages. However, closer examination indicates that in an A-B block schedule hits 
were recorded for each day of the school week. This looms important as it suggests that in some way students 
gained access to the shared Web-based environment during days when no science class was scheduled. 



Discussion Forum Findings 

Analysis of the use of the shared environment’s discussion board revealed much about the amount of time 
spent using a shared discussion place as well as about other previous discussion forum experiences within 
the classroom. Extrapolation of the number of hits associated with accessing the discussion board provided 
evidence of minimal use of this tjqie of environment and technology, suggested a lack of previous exposure to 
the electronic sharing of ideas and information, and detected technical issues as the most pressing and 
reported discussion topics. The following sections address these extrapolations. 
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Date of Use 


Day of the 
Month 


Scheduled 
Study Dates 


Hits 


Percentage of 
Hits (%) 


5-4-98 


Monday 


+ 


4 


1.4 


5-5-98 


Tuesday 




7 


2.5 


5-6-98 


Wednesday 


+ 


7 


2.5 


5-7-98 


Thursday 




8 


2.8 


5-8-98 


Friday 


+ 


103 


39.2 


5-11-98 


Monday 




70 


24.7 


5-12-98 


Tuesday 


+ 


16 


5.7 


5-13-98 


Wednesday 




27 


9.5 


5-14-98 


Thursday 




25 


8.8 


5-15-98 


Friday 




16 


5.7 


Total 






283* 


100.0 



Note:The * denotes the number of hits remaining after discounting the number of hits recorded by the 
dissertation’s author while monitoring the site. 



Use of the shared discussion board 

Originally, a total count of discussion threads posted by the author of the study was determined. Sixteen 
threads were initially established as “starters” for each of the activities in the ecology curriculum guide. The 
use of starter threads potentially enabled moderation and enumeration of topics by the author or by each 
campus teacher. A count of the number of discussion responses indicated very little activity within the shared 
discussion board. Minimal new discussion threads were originated through the duration of the study. The 
lack of discussion threads or responses seemed surprisingly low considering that each curriculum activity 
included responses to the discussion board topics as indicated in each activity directions. Analysis of 
discussion threads and responses suggested little time of use within the forum opportunity of the site. Only 
five new discussion threads were recorded over the duration of the study. Of the new threads recorded only 
one came from a student. Efforts to understand this lack of activity brought focus to the nature of the 
discussion threads or response messages actually recorded within the discussion forum. 



Nature of the discussion forum threads and responses 

A review of the content of the discussion threads created by teachers and students indicated their experiences 
with technical difficulties associated with some aspect of the various technologies involved in the study. 
Primarily, use of the uploading and downloading capabilities presented time-consuming difficulties as 
apparent in the number of discussion threads and responses made by teachers and the one student 
discussion response dealing with this aspect of the study. Additionally, qualitative analysis of the dissertation 
author’s discussion responses suggests frustrations and needed help on the part of the writers of the 
responses. In some instances the discussion thread titles indicated the degree of technical problems being 
experienced: “Feeling Frustrated-Punch Here,” “Need Directions,” or “Need Directions too.” 

An analysis of the dates of the forum threads and responses provides additional evidence of the minimal use 
of the discussion forum as well as potential reasons for this lack of use. The dates of teacher and student 
discussion threads and responses seemed to indicate initial use occurred late in the study and did not follow 
the curriculum guide of activities. Table 2 summarizes discussion threads and responses by date. 
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Table 2 seems to suggest that the teachers, at least, were beginning to try to use the forum as a place to share 
difficulties as well as a place for seeking help. When dissertation author discussion threads and responses 
were analyzed, all appeared to address technical issues inherent in the software or in the computer platform 
providing technology capabilities. 



Table 2. Summary of Discussion Threads and Responses by Date 



Discussion Thread Titles 


Date 


Respondent 


Test Discussion 


May 1 


Teacher 


Test Run 


May 1 


Teacher 


Feeling Frustrated-Punch Here 


May 8 


Teacher 


Need Directions 


May 11 


Student 


Need Directions too 


May 11 


Teacher 


Discussion Response Titles 
Re: Ditch Site Locations 


May 5 


Teacher 



Difficulties Experienced Using Comseinfo Software 

Examination of uploaded files and discussion forum transcripts provided evidence of a variety of difficulties 
experienced with the piloted software. These difficulties seemed to cluster around novice user levels of the 
participants, computer platform inconsistencies, and software security issues. 

When an initial survey of all uploads in to the Courseinfo ditch ecology site was performed, upload attempts 
provided evidence that use of the site by students was high. However, this same information suggested that 
the on-site help pages and on-hand training documents had not been accessed during initial uploads, as 
many attempts to upload were to incorrect locations. 

Once user uploads were found, attempts to open and view each was made. This was initially undertaken by 
the dissertation author to ensure the viewability of the data. Computer platform issues came into evidence 
during this phase for all campus users. All campus users operated Macintosh platforms that automatically 
upload files with no preassigned file extension added to the file name. PC platforms and software 
automatically preassign the file type extension to a file name, thereby enabling identification of the file type. 
This became critical within the Courseinfo environment as each uploaded file, when opened, launched a copy 
of the application needed for readability. The lack of automatic assignment of file type extensions as a result 
of Macintosh platform use caused initial uploads by each campus to be unreadable. This issue created 
multiple upload attempts of the same data and high frustration levels among the users uploading. 



Summary 

The secondary research question attempts to identify academic behaviors that can be tracked within a shared 
Web-based learning environment such as that offered by the piloted Courseinfo software. Identifying user 
movement patterns in Web-based environments and discussion forum patterns as well as attempting to 
ascribe meaning to these movement patterns by connecting these movements to academic problem-solving 
and metacognitive reflection offers much potential for understanding these new learning environments. 
Connecting meaning to these movement trends within the context of the shared Web-based environment 
may be the most interesting, potentially meaningful aspect of this study. 
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As shared Web-based learning environments within secondary classrooms experience rising use, models 
focused on understanding the critical elements of effectiveness of these environments seem necessary, rather 
than novel. Tracking user movement and discussion forum patterns offers a potential way of documenting 
and quantifying learner critical thinking and metacognitive reflection within a Web-based learning 
environment. Tracking movements and discussion forum patterns provides one method of assessment 
within the Web-based environment that can be maintained behind the scenes without interference to the 
learner and without overt, game-like feedback. This type of assessment informs instruction, the instructor, 
and the learning environment. The potential exists for measuring the effect of the environment through the 
learner’s movement patterns, thereby offering the possibility of creating an optimum learning environment 
for each and every type of learner. 

The richness of shared data, data examination, and learner insights follows naturally from analyzing user 
movement patterns. Moving learner metacognitive reflection capabilities to a Web-based learning 
environment presents the academic accountability often missing from current models of Web-based 
learning. Not only depth of knowledge, but also critical-thinking levels, can begin to be warehoused for 
constructive use by each learner/user. 

Contextualizing Web-based learning environments with real-time scientific data collection and evaluative 
activities appears to favorably support the improvement of academic behaviors. Given the strong emergence 
of distance education initiatives, evaluating and assessing these behaviors within the Web-based learning 
environment becomes ever more important. Add to this the constructivist nature associated with this study’s 
model, and the power and empowerment of this type of Web-based learning place becomes awesome, and 
needed. 



Future Implications 

The results of this study present one practical model for infusing technology in to the classroom setting; for 
improving problem-solving ability and metacognitive reflection over a short duration; for creating a 
collaborative, cooperative learning space; and for maintaining a science space for learning where no gender 
differences arise. 

The power an intranet offers within the constraints of a school district, or geographic locale, has not yet been 
tapped. This study proposes one mechanism for doing just that given the infrastructure present or absent 
through the use of a Web-based intranet. This model offers a “get your feet wet" method of networked 
connectivity for classrooms and teachers who have not yet jumped in to the World Wide Web. 

This research provides a study in contextualizing connectivity with end goals of improved problem-solving 
and metacognitive reflection. Both of these elements are often lost when initial attempts to jump in to 
networked learning occur or are contemplated. Further, this study provides an avenue of documenting the 
nature of learning during the use of Web browsing or other networked connections. Tracking learner 
movements within a browsed Web site has metacognitive as well as problem-solving implications for each 
and every learner. 
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